Private Weighted Random Walk Stochastic Gradient Descent

We consider a decentralized learning setting in which data is distributed over nodes graph. The goal to learn global model on the without involving any central entity that needs be trusted. While gossip-based stochastic gradient descent (SGD) can used achieve this objective, it incurs high communication and computation costs, since has wait for all local models at converge. To speed up convergence, we propose instead study random walk based SGD updated two algorithms types of walks achieve, way, uniform sampling importance data. provide non-asymptotic analysis rate taking into account constants related Our numerical results show weighted algorithm better performance high-variance Moreover, privacy-preserving achieves differential privacy Gamma noise mechanism propose. also give convergence outperforms additive Laplace-based mechanisms.